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This study compared 34 lower-achieving (developmental) first-time college 
students' self-reported self-regulation strategies from a Likert scale to those 
they reported in struetured interviews. Likert scales have offered eonvenient 
administration and evaluation and have been used to identify what and 
how learners study. The reported study activity of regular admission college 
students' has predicted their subsequent college achievement; in the same 
study, responses did not support concurrent validity for the lower-achieving 
students (Nist, Mealey, Simpson, S'Eroc, 1990). College students who failed 
to meet regular admission requirements enrolled in 2 sections of a college 
study skills course reported significantly different strategy use between their 
Likert and interview responses on five of the fifteen strategies classified by 
each of the measures. 

T) 

^k.esearchers and practitioners have 
used Likert scales to measure learning processes and activities that are 
generally regarded as part of self-regulation (SR) in that they measure 
activities students employ to engage, monitor, sustain, and evaluate their 
learning processes (cf, Nist, Mealey, Simpson, & Kroc, 1990; Zimmer- 
man S" Martinez-Pons, 1988). Self-regulation, synonymous with metacog- 
nition (Brown, Hedberg, & Harper, 1994) or metacognitive adjustments 
by learners in response to feedback on errors (Brown, Bransford, Fer- 
rara, & Campione, 1983), has been extended to activities that support 
self-regulation, such as managing distractions (Orange, 1999). Evidence 
indicates that selected SR strategies may be a key distinguishing charac- 
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teristic between developmental and regular admission college students 
and between more and less expert learners (Ley & Young, 2001). The 
current study was designed to add to the literature on which learning 
processes, specihcally self-regulation activities, developmental students 
use, with what frequency they are used, and to conhrm comparability of 
two self regulation measures with developmental students, an interview 
and a Likert scale. 

The National Center for Education Statistics (NCES, 2001) reports 
that 71 % of all 4-year institutions and 80 % of 2-year institutions offer 
some form of remediation. Educators assisting these students attempt 
to identify their learning processes through the use of self report 
measures, largely Likert scale items. University educators have used 
Likert measures of study activities to advise students how to improve 
their study habits. While researchers have used Likert and structured 
interviews to identify students’ learning processes, colleges have used a 
student’s responses to Likert measures as an advising tool. Researchers 
investigating self-regulation want unbiased data to interpret and from 
which to draw conclusions. Conhrming the measures’ comparability 
would provide support using a valid, easily-administered and interpreted 
Likert scale with lower-achieving college students. Furthermore, “if 
at-risk [college] students are to be identihed. . . the literature suggests 
that tailored interventions can boost students’ success" (Le, Casillas, & 
Robbins, 2005, p. 502). 

Ambiguous results from earlier studies using self-report Likert mea- 
sures to identify study activities and their frequency have raised ques- 
tions about using such measures with lower-achievers, especially college 
developmental students (Deming, Valeri-Gold, & Idelman, 1994; Ley 
Young, 1998b; Nist et ah, 1990; Young & Ley, 1997). A combination of ten 
different Likert scales to measure learning processes predicted college 
GPA for regular admission students but not for developmental students 
(Nist et ak, 1990). Likert responses predicted achievement among 168 
regular admission freshmen in a study skills course but not among 71 
developmental freshmen in the same course. No combination of the 
ten study-skill scale scores regressed upon developmental student CPAs 
scales was predictive. Developmental and regular admission students 
from intact classes reported using the same study activities with the same 
frequency on the four factors: repetition, integration, comprehension 
monitoring, and coping (Ley & Young, 1998a). 

On the other hand. Ley and Young (1998b) found that when respond- 
ing to an interview, developmental students reported using signihcantly 
fewer strategies and using them less often than did regular admission 
students. A discriminant analysis of 15 self-regulation strategies from 
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the interview correctly classified 73% or 21 of the 28 regular admission 
students and 25 of the 31 developmental students. The evidence to reject 
a Likert scale is far from conclusive. This investigation provides data to 
identify what strategies developmental students report using and how 
often. The data add to the literature on self-regulation and relative use- 
fulness of two different measures of self-regulation. If lower-achievers’ 
Likert responses align with their interview reports, they would cross 
validate each other. An investigation comparing the two types of scales 
would also contribute to the body of research to inform policy decisions 
at institutions using Likert scales to diagnose developmental student’s 
study and learning strategies. This study used two different measures 
of self-report, a Likert scale, and a structured interview, to capture the 
types of strategies reportedly used by developmental college students. 

Method 

This study compared developmental students’ strategy frequency 
rankings from interviews to those from Likert items. Lower-achieving 
college students reported the activities they would complete and how 
frequently they used the strategies when presented with each of eight 
instructional circumstances in a structured interview. Within two weeks 
of completing the interview, the same students ranked the frequency 
with which he or she used fifteen self-regulation strategies presented 
as Likert statements. 

Sample 

Participants were postsecondary first semester students enrolled in two 
sections of a college study skills course offered at a two-year institu- 
tion in a south-central location of the United States. The students were 
enrolled in the study skills course based upon an ACT score below 18, 
the minimum for regular admission. The mean age for the 35 partici- 
pants was 21.7 with 63% between 17 and 19 years of age. Participants 
had a mean ACT composite score of 13.5. 'There were 12 males and 23 
females and 19 whites, 13 African Americans, and three other race or 
race not identified. 

Measures 

Data were collected from participants with each of two measures, a 
fifteen-item Likert scale (Lan, 1996, Appendix A) and an eight-item 
structured interview (Zimmerman Sr Martinez-Pons, 1988, Appendix 
B). The Likert-scaled items present each of the fourteen strategies as 
an activity for which the respondent ranks the frequency with which 
he or she uses the strategy. 'The fourteen strategies represent a fairly 
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comprehensive schema for classifying learner self-regulating activities: 
self-evaluation, organizing and transforming, goal-setting and planning, 
seeking information, keeping records and monitoring, environmental 
structuring, self-consequences, rehearsing and memorization, seeking 
assistance from teachers/experts, from classmates, or from others, 
reviewing tests, reviewing notes, reviewing texts (Zimmerman & Mar- 
tinez-Pons, 1986). The hfteenth category is reserved for strategies other 
than self-regulation strategies. 

Zimmerman and Martinez-Pons (1986) developed the structured inter- 
view instrument to elicit self-regulated strategies but without prompting 
participants with specihc strategies. Using a post hoc design, 93% of 
80 high school participants were correctly classihed into preassigned 
high/low achievement tracks based on strategy use as measured by the 
interview. In a subsequent study with participants from hfth, eighth, 
and eleventh grades, they found that students who were classihed as 
gifted reported statistically signihcant greater use of self-regulated 
learning strategies than did students classihed as regular (Zimmerman 
& Martinez-Pons, 1990). 

The structured interview protocol (Zimmerman & Martinez-Pons, 
1986, Appendix B) requires the interviewer to ask the participant ques- 
tions for which an interviewer records the strategy and the frequency 
indicated by the interview. The interviewer poses open-ended ques- 
tions about what strategy the participant would use in eight different 
instructional circumstances. The interviewer always must ask for and 
record additional strategies the student uses when confronted with the 
situation described in the item. In contrast a student completing the 
Likert items indicates the frequency with which he or she engages in 
each of 15 specihc activities, such as “seeking assistance from teach- 
ers/experts." 

We adapted the interview to postsecondary participants by deleting the 
words “report card” before the word “grade” in two of the eight learning 
instructional circumstances, substituting the words “the semester” for 
“marking periods” in one learning instructional circumstance and delet- 
ing the words “your family's” before the word “history” in one learning 
instructional circumstance. These were the only changes to the protocol 
developed by Zimmerman and Martinez-Pons (1986). Both measures 
yield hve ordinal frequency ranks for each of 15 self-regulation strate- 
gies. Likert and Interview frequency terms were assigned numerical 
ranks from 0 to 4, corresponding from least to most frequent responses. 
Thble 1 lists the frequency options by measure and the resulting recoded 
numerical value for each of the hve different frequency terms from 
each measure. 
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Thble 1 

Mode, Median and Pereentile Rank for Self-Regulation (SR) Strategy 
Use by Likert and Interview Measures (N = 34) 


Strategy Item 

n 

Mdn 

Mode 

Percentiles 






25% 

50% 

75% 

Likl Self-Evaluation 

34 

3.0 

3 

2.0 

3.0 

3.3 

Inti Self-Evaluation 

35 

4.0 

4 

3.3 

4.0 

4.0 

Lik2. Organizing and transforming 

34 

3.0 

4 

2.0 

3.0 

4.0 

Int2 Organizing and transforming 

35 

3.0 

0 

0.0 

3.0 

4.0 

Lik3 Goal-setting and planning 

34 

3.0 

3 

2.0 

3.0 

3.0 

Int3 Goal-setting and planning 

35 

0.0 

0 

0.0 

0.0 

3.7 

Lik4 Seeking information 

34 

3.0 

4 

2.0 

3.0 

4.0 

Int4 Seeking information 

35 

2.3 

0 

0.0 

2.3 

3.5 

Lik5 Keeping records & monitoring 

34 

3.0 

3 

2.0 

3.0 

4.0 

IntS Keeping records S' monitoring 

35 

3.0 

4 

0.0 

3.0 

4.0 

Lik6 Environmental structuring 

34 

3.0 

4 

2.0 

3.0 

4.0 

Int6 Environmental structuring 

35 

3.2 

4 

2.7 

3.2 

3.7 

Lik7 Self-consequences 

34 

2.0 

2 

1.0 

2.0 

2.0 

Int7 Self-consequences 

35 

0.0 

0 

0.0 

0.0 

3.0 

Lik8 Rehearsing & memorization 

34 

3.0 

3 

2.0 

3.0 

4.0 

Int8 Rehearsing & memorization 

35 

0.0 

0 

0.0 

0.0 

3.0 

Lik9 Seeking assist from teachers 

33 

3.0 

2 

2.0 

3.0 

4.0 

Int9 Seeking assist from teachers 

35 

2.0 

0 

0.0 

2.0 

3.0 

LiklO Seeking assist from classmates 

34 

2.0 

2 

2.0 

2.0 

3.0 

IntlO Seeking assist from classmates 

35 

0.0 

0 

0.0 

0.0 

3.0 

Likll Seeking assist from others 

34 

2.0 

2 

2.0 

2.0 

3.0 

Intll Seeking assist from others 

35 

3.0 

4 

1.0 

3.0 

4.0 

Likl 2 Reviewing tests 

34 

3.0 

4 

2.0 

3.0 

4.0 

Inti 2 Reviewing tests 

35 

3.5 

4 

3.0 

3.5 

4.0 

Likl 3 Reviewing notes 

34 

3.5 

4 

3.0 

3.5 

4.0 

Inti 3 Reviewing notes 

35 

0.0 

0 

0.0 

0.0 

0.0 

Likl 4 Reviewing texts 

34 

3.0 

3 

2.0 

3.0 

3.0 

Inti 4 Reviewing texts 

35 

2.0 

0 

0.0 

2.0 

4.0 

Likl 5 Other 

33 

2.0 

3 

2.0 

2.0 

3.0 

Intis Other 

35 

3.6 

4 

3.0 

3.6 

4.0 


Note: Frequency responses ranked from 0 for none mentioned (Lik) or never 
(Int) to five for always (Lik) or most-of-the-time (Int). 
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Since the Likert scale and the interview measure are not summed 
scores but rank orders of heterogeneous items, calculating reliability 
estimates for either of the measures are not appropriate (cf , Pedhazur 
& Smelkin, 1991). Each Likert item and the interview yield a frequency 
classihcation for each of 14 different self-regulation activities and one 
other category. The correlations between higher self-regulation activity 
levels and higher achievement levels support the validity of the Likert 
measure (Lan, 1996) and the structured interview (Zimmerman & Mar- 
tinez-Pons, 1986) with regular- or high-achieving students. 

Procedures 

A graduate research assistant trained to conduct the interviews went to 
the two classes and asked participants to voluntarily participate in the 
study. Those who agreed signed an informed consent form, completed 
a demographic data form, and were interviewed during the hrst two 
weeks of the semester. During the last class of the hrst two weeks in the 
semester and after completing the interviews, participants completed 
the Likert scale. Tb avoid the response cuing inherent in the Likert, par- 
ticipants were interviewed prior to receiving and completing the Likert 
scale. The interview does partially cue the respondent to the degree that 
it describes conditions during which many learners employ a particular 
self-regulation strategy, although none is implied or suggested. The 
Likert was administered to the group of participants in each class after 
the interviews were completed for the class. 

Following the protocol validated by Zimmerman and Martinez-Pons 
(1986), the interviewer read eight different instructional circumstances 
to each participant and then asked questions about what the participant 
did in that circumstance (Appendix B). For example, one circumstance 
was 'Assume a teacher is discussing a topic with your class, such as the 
history of the civil rights movement. He or she says that you will be 
tested on the topic. Do you have a particular method to help you learn 
and remember what was discussed in class?” After the participant told 
the interviewer what he or she would do, the interviewer then asked, 
"What if you are having difficulty, is there any particular method you 
use?” The interviewer recorded what the participant said on a form. 
Regardless of whether or not the interviewee offered one or more strat- 
egies for an instructional circumstance the interviewer asked, "What 
if you are having difficulty? Is there any particular method you use?” 
(Zimmerman & Martinez-Pons, 1986, p. 619). The participant was asked 
how consistently (seldom, occasionally, frequently, or most of the time) 
he or she uses a strategy that he or she had previously identihed. For 
example, the interviewer asked the participant how often do you "read 
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the book— 1, seldom; 2, occasionally; 3, frequently; or 4, most of the 
time?” The participant, who held a card printed with the same rating 
options, rated the consistency with which he or she used the strategy 
and the interviewer recorded the consistency number by the strategy. 

TWo trained raters classihed each instructional circumstance response 
as one of 15 self-regulating strategy categories: self-evaluation, organiz- 
ing and transforming, goal-setting and planning, seeking information, 
keeping records and monitoring, environmental structuring, self-conse- 
quences, rehearsing and memorization, seeking assistance from teach- 
ers/ experts, from classmates, or from others, reviewing tests, reviewing 
notes, reviewing texts, and other [than SR], Every response was coded 
independently and fewer than 5% were discrepant and those were 
quickly reconciled through a brief discussion between the two raters. 

The interview results yielded multiple frequency ranks for a strategy 
category, because the participant could identify more than one strategy 
or activity to the interviewer. The Likert measure resulted in only one 
frequency . Strategies for which participants did not identify any activity 
were coded with a 0 frequency; the equivalent Likert was ranked never. 
If the participant did not mention a strategy in a particular category 
during the interview, the frequency was 0. The frequency rank for the 
interview self-regulation strategy categories was calculated by averaging 
the frequency ranks given for each activity in the same self-regulation 
category. Therefore if a participant only mentioned one activity in a 
category during the interview, the frequency for that activity was the 
self-regulation category frequency. If a participant identihed two or more 
activities in the same self-regulation category, the average frequency 
for the two or more activities represented the interview self-regulation 
category frequency that was compared to the corresponding Likert 
self-regulation category frequency. Because only 4 of the interview 
self-regulation frequency ranks had a mode of 2 or greater and 9 of 15 
interview self-regulation frequency ranks had a mode of 0 or 1 (the other 
category does not contain self-regulation strategies), the potential for 
distorting the data through averaging two or more frequencies in the 
same category was limited. 

Results 

Histograms revealed a threat to the normal distribution assumption 
for parametric t-tests. The Lilliefors test statistic, a variation of the Kol- 
mogorov test when estimating parameters from the data, determined 
if the values came from a normal distribution (Sprent, 1989). None of 
the interview or Likert frequency rank tests had normal distributions. 
Therefore, nonparametric tests were more appropriate because of the 



Self-regulation measures 


67 


ordinal nature of the data and because the data were not normally dis- 
tributed, although these tests are less powerful than the corresponding 
t test for normally distributed, interval data (Gall, Borg, & Gall, 1996). 
Items 1,3,7, 8, and 10 met the assumptions for a Wilcoxon signed ranks 
test (Daniel, 1978; Easton Gr McColl, 1997) which assumes a symmetrical 
distribution for the differences between the two ranks to calculate the 
test statistic. 

Each strategy was analyzed individually to enable comparisons for 
each strategy so that they would not be masked by grouping responses 
together for ordinal data. Compensating for the threat to Type I error 
rate inflation possible with 15 pair-wise ordinal comparisons, the level 
of signihcance for each test was set atp = .01. The current data met as- 
sumptions for a paired signs test; both measured a continuous variable 
on an ordinal scale with independent values for each self-regulation 
measure taken from the same subject (Daniel, 1978). The Wilcoxon test 
ranked the differences, excluding no-difference pairs before assigning a 
sign and summing the positive and negative ranks to determine if the 
median difference was zero. The sign test was calculated by summing 
all the positives, negatives, and ties and then computing a Z score and 
a value associated with the frequency of the positives and the negatives 
(George & Mallery, 1999). A binomial test occurs when there are two 
observations possible, with and without a certain characteristic, each 
with a probability of one-half; otherwise, test is the same as a sign-test. 
The tests comparing the hfteen strategy pairs revealed the consistency 
or discrepancy between the Likert and self-regulation interview strategy 
frequency rankings of the responses offered by the participants (Thble 
2 ). 

The nonparametric measures of central tendency revealed more 
evenly distributed responses within the Likert frequencies (Thble 1) than 
in the interview frequencies. If participants identihed a strategy during 
an interview they reported using the strategy more frequently than when 
ranking the frequency on the Likert. Interview strategy frequency rank 
modes equaled either the lowest or the highest of the potential value; 
nine were 0 and six were 4. Seven of the 10 interview strategies had me- 
dians between 3, frequently, and 4, most of the time; hve were 0; three 
were between 2 and 3. Likert responses had a mode 2 or greater for all 
strategy categories and were more evenly distributed with modes of 2 
for four strategies, 3 for six strategies, and 4 for hve strategies. 

Nine of the interview strategies were often not mentioned at all; hve 
of these had modes and medians of zero (Items 3, 7, 8, 10, 13). The nine 
interview strategies with a mode of 0 included organizing and trans- 
forming (Strategy 2), goal-setting and planning (Strategy 3), seeking 
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Thble 2 


Nonparametric Ttsts for Negative and Positive Differenees 




Z’ 

Differences 

Negative 

Differences 

Positive 

Ties 

1 

Self-Evaluation“ 

-3.39*** 

6 

25 

3 

2 

Organizing and 
transforming’’ 

-1.49 

19 

10 

5 

3 

Goal-setting and 
planning” 

-3.32*** 

20 

9 

5 

4 

Seeking information’’ 

-2.55 

20 

6 

8 

5 

Keeping records and 
monitoring “ 

n/a 

13 

9 

12 

6 

Environmental 

structuring’’ 

-0.39 

12 

15 

7 

7 

Self-consequences” 

-1.49 

18 

10 

6 

8 

Rehearsing and 
memorization” 

-4.11**** 

27 

4 

3 

9 

Seeking assist from 
teachers “ 

n/a 

20 

4 

9 

10 

Seeking assist from 
classmates” 

-2.72 

21 

7 

6 

11 

Seeking assist from 
others “ 

n/a 

11 

14 

9 

12 

Reviewing tests “ 

n/a 

7 

14 

13 

13 

Reviewing notes” 

-4.69**** 

28 

0 

6 

14 

Reviewing texts’” 

-1.44 

20 

11 

3 

15 

Other’’ 

-3.47*** 

5 

25 

3 


‘‘Wilcoxon signed rank test; Sign test; “sign test for binomial distribution. 

***p < .001; ****p < . 0001 

information (Strategy 4), self consequences (Strategy 7), rehearsing and 
memorization (Strategy 8), seeking assistance from teachers (Strategy 9), 
seeking assistance from peers (Strategy 10), reviewing notes (Strategy 
13), reviewing texts (Strategy 14). The remaining hve interview modes 
had the highest possible value, 4, self-evaluation (Strategy 1), keeping 
records and monitoring (Strategy 5), environmental structuring (Strategy 
6), seeking assist from peers (Strategy 11), and reviewing tests (Strategy 
12). The remaining mode, other (Strategy 15), was for strategies that 
students reported using that were not self-regulation (e.g., getting a 
drink, praying). 
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Most Likert strategy measures of central tendency were between 2 
and 4. Likert medians for self-consequences (Strategy 7) equaled 3 and 
seeking assistance from classmates or others (Items 10 & 11) and using 
other [than SR] (Strategy 15) equaled 2 the same as the mode. Likert 
median values also were more central or moderate frequency than were 
the interview medians; the Likert medians were at least 2, sometimes, to 
3.5, more than often whereas the interview medians were 0, strategies 
not mentioned, to 4, strategies used most-of-the-time. 

Likert medians and modes exceeded interview medians and modes 
for eight strategies: Goal-setting and planning (Strategy 3), seeking in- 
formation (Strategy 4), self consequences (Strategy 7), rehearsing and 
memorization (Strategy 8), seeking assistance from teachers (Strategy 
9), seeking assistance from peers (Strategy 10), reviewing notes (Strat- 
egy 13), reviewing texts (Strategy 14). In contrast, interview median 
and mode were higher than Likert median and mode for strategies 
self-evaluation (Strategy 1), seeking assist from peers (Strategy 11), and 
other (Strategy 15). 

For hve strategy categories, the interview frequency ranks were higher 
than the Likert frequency ranks: self-evaluation (Strategy 1), environ- 
mental structuring (Strategy 6), seeking assist from peers (Strategy 11), 
reviewing tests (Strategy 12), and the other [than SR] category (Strategy 
15). The remaining 10 had negative differences, indicating they used 
the strategy more often when responding to the Likert than during the 
interview: organizing and transforming (Strategy 2), goal-setting and 
planning (Strategy 3), seeking information (Strategy 4), keeping records 
and monitoring (Strategy 5), self consequences (Strategy 7), rehearsing 
and memorization (Strategy 8), seeking assistance from teachers (Strat- 
egy 9), seeking assistance from peers (Strategy 10), reviewing notes 
(Strategy 13), reviewing texts (Strategy 14). Ttvo strategies had almost a 
third of the participants report the same frequency for the same strategy 
on each measure. Records and monitoring (Strategy 5) and reviewing 
tests (Strategy 12), tied 12 and 13 times respectively; both strategies had 
medians and modes equal or greater than 3. 

The interview and Likert responses did not differ signihcantly for 
ten of the hfteen measures, thereby offering some support for their 
comparability and concurrent validity. The Wilcoxon test identihed 
hve strategy frequencies (self-evaluation, goal-setting..., rehearsing..., 
reviewing notes) that differed signihcantly; the other category differed 
although it is not a strategy. The Likert scale elicited a higher frequency 
response for all but self-evaluation. The highest Likert medians were 
reported for 13 (3.5), and strategy 2, 4, 6, 12, 1, 3, 5, 8, and 9 with a 3; 
interview, 1 (4), 15 (3.6), 12 (3.5), 6 (3.2), and 5, 11, and 2 all with 3. 
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The means from the ten items not signihcantly different between the 
interview and Likert responses indicate the same frequency for strate- 
gies. Tb determine which strategies students used most frequently, the 
frequency means for Likert and interview, were sequenced so that both 
interview and Likert means for any item precede any other item means 
that follow. The ten, from most to least frequent followed by the inter- 
view and Likert means in parenthesis are reviewing tests (Strategy 12; 
3.15, 2.94), environmental structuring (Strategy 6; 3.06, 2.91), seeking 
assist from others (Strategy 11 ; 2.51 , 2.47), keeping records and monitor- 
ing (Strategy 5; 2.45, 2.85), organizing and transforming (Strategy 2; 2.20, 
2.79), seeking information (Strategy 4; 1.97, 3.06), seeking assistance 
from teachers (Strategy 9; 1.95, 2.91), reviewing texts (Strategy 14; 1.94, 
2.91), assistance from peers (Strategy 10; 1.34, 2.26), self-consequences 
(Strategy 7; 1.2, 1.71). 

There were statistically signihcant differences in rank orders between 
the Likert scale and the interview frequencies for hve of the hfteen 
strategy categories. The signs test answered one research question, did 
developmental participants report different frequency ranks for a self- 
regulation strategy recorded on the Likert measure than reported during 
a structured interview? The p value of the Wilcoxon Signed Ranks tests 
were set at .01 to determine the percentage of the time that the test would 
incorrectly indicate a statistically signihcant difference in the number 
of positive and negative discrepancies between the two measures. The 
two different measures elicited a sufficient number of plus or minus 
signs to indicate a statistically signihcant difference between the two 
distributions of self-evaluation (Strategy 1), rehearsing and memoriza- 
tion (Strategy 8), reviewing notes (Strategy 13), and the other [than SR] 
category (Strategy 15); Likert values were higher for Strategies 1 and 
15 and lower for the other two. The sample differences between the 
Likert scale and the interview matched the pair values for these four 
categories. The sign tests did not detect the magnitude of the discrep- 
ancy differences, only a statistically signihcant number of either higher 
or lower rankings, and, therefore, effect size of the difference could not 
be determined. Spearman’s rho correlations tested for the strength of 
the relationship between items for the paired frequencies; none of the 
paired correlations were signihcant at the 0.1 level. 

Tb further determine the validity of the two alternative forms of self 
report for each of the hfteen strategy categories, a Kendall’s rank corre- 
lation statistic tested for a relationship between a participant’s reported 
strategy use and his or her achievement, as none of the frequency ranks 
in any of the hfteen categories in either measure nor the achievement 
scores conformed to a normal distribution. There was no correlation 



Self-regulation measures 


71 


between course points and frequency ranks in any of the fifteen strategy 
use categories from either of the two self-report measures, the interview 
or Likert. The total number of different strategy categories participants 
suggested was also uncorrelated with total course points. Only the total 
number of different activities for the interview self-evaluation (Strategy 
1) strategy category, was moderately correlated to total course points 
(n = 33, T = .33, p < .05). 

Developmental student achievement was unrelated to any self-re- 
ported self-regulation strategies indicated by the Likert or through the 
interview. The achievement measure, course points, may have lacked 
range to detect any differences for any one measure; the total points 
possible with bonus points was 1060; over half the participants earned 
over 950 points and 85% earned over 730 points. The relatively flat dis- 
tribution clustered at the high-end indicated that most of the participants 
in this class earned most of the points possible. Therefore the lack of 
relationship between achievement and self-regulation maybe explained 
partially by the clustering of data points at the high end of the scale. 

Discussion 

There was some support for using the results from a Likert scale to 
identify and assist developmental, that is lower-achieving or at risk, 
students. Responses to the two measures did not signihcantly differ 
for ten of the hfteen strategy frequencies: organizing and transforming 
(Strategy 2), keeping records and monitoring (Strategy 5), environmental 
structuring (Strategy 6), consequences (Strategy 7), seeking assistance 
from teachers (Strategy 9), seeking assistance from peers (Strategy 10), 
seeking assist from peers (Strategy 11), reviewing tests (Strategy 12), 
and reviewing texts (Strategy 14). On the other hand, developmental 
students reported using hve strategies more often on the Likert scale 
than during the interview: goal-setting and planning; seeking informa- 
tion; rehearsing and memorization; and reviewing notes. 

Three Likert items elicited signihcantly higher strategy frequencies 
than did the interview. Results from three of the Likert items should 
be cautiously interpreted: goal-setting and planning, rehearsing and 
memorization, and reviewing notes. At least half of the respondents 
never mentioned using three of the hve strategies during the interview: 
goal-setting and planning; rehearsing and memorization; and review- 
ing notes. Without a Likert item to cue them, many learners did not 
report any activities for these three strategies; the median and mode 
were zero. These lower-achieving learners may have been offering 
socially acceptable responses to the Likert scale items for goal-setting 
and planning; seeking information; rehearsing and memorization; and 
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reviewing notes, since in all cases many of the students did not report 
using the strategy at all during an interview but did when prompted by a 
Likert item including the strategy. Higher and lower achieving students 
reported comparable strategy frequencies for these three items in an 
earlier study using the interview (Ley & Young, 1998b), which may be 
more accurate. 

The only self-regulation strategy developmental learners reported us- 
ing more often in an interview than in a Likert item was self-evaluation. 
Only one of the interviewees did not mention some form self-evalua- 
tion, evidence that these students do practice self evaluation (checking 
their own work). The interview median and mode at highest frequency, 
4, contrasted to the Likert median and mode at 3. Perhaps since the 
interview activities were reported and then coded by researchers, the 
learners did not recognize self-evaluation as synonymous with activities 
they described in the interview earlier. For example, the student may 
not recognize an activity such as, “I look at my assignment to be sure 
it is compete before I turn it in". It is also possible that self-evaluation 
is a SR strategy used by developmental learners, however, it is a strat- 
egy that requires completion of activities to be evaluated, and would 
be insufficient as an effective learning strategy independent of other 
applied strategies. 

Developmental students have many activities they claim to engage in 
while studying that are unrelated to studying; they may be considered 
study distractions, such as getting a drink. Interviews elicited greater 
frequency for activities that were not self-regulation activities at all; they 
were classihed as other [than self-regulation]. The interview median and 
mean frequency for the distractions were at least one point higher than 
for the Likert. The interview may collect more accurate details about 
such misconceptions about what a study strategy is and how often a 
student uses activities counterproductive to learning while considering 
them study activities. 

Four activities they reported using most often were among the strat- 
egies which have discriminated between developmental and regular 
admission students. Despite the fact they report using them most often 
in this study, developmental students in an earlier study reported using 
them less often in an interview than did the regular admission students 
(Ley & Young, 1998b). The activities, reviewing tests, environmental 
structuring, keeping records and monitoring, and organizing and trans- 
forming materials, have predicted higher achievement status. This may 
mean that they do not effectively implement these strategies but know 
that they should be using them; or that they use them often but not 
as often as required for academic success. Those that were used least 
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often, seeking information and assistance from teachers and peers, con- 
trasted with seeking information from others, which was the third most 
frequently reported activity as measured by means of the ten strategies 
which do not differ on Likert and interview measures. Developmental 
educators may want to encourage students to seek help from experts, 
such as their instructors or tutors, rather than asking someone else about 
how to complete work. 

There was evidence impugning the validity ofboth measures. Neither 
the Likert nor the interview strategy frequencies correlated with the 
developmental students’ total course points. The interview and Likert 
scale item results for developmental students at the beginning of the 
semester were unrelated to their subsequent course achievement. On the 
other hand, lack of any statistically signihcant relationship with achieve- 
ment maybe an artifact of restricted range for the achievement measure 
and the lack of power of the nonparametric test. Likert self-reports of 
learning and study strategies have not been related to developmental 
student success but have been related to regular admission student 
college success (Deming et ak, 1994; Nist et ah, 1990). Interview data 
collected with the same protocol had successfully predicted develop- 
mental or regular admission classihcation, a dichotomous achievement 
indicator (Ley & Young, 1998b). 

During the interviews, most developmental students offered "other” 
strategies, (such as, get a drink, just do it, or pray, obviously not self-regu- 
latory strategies. Tlventy-hve respondents had ranked "other" interview 
frequency above the Likert frequency. Developmental participants were 
able generate several non-self-regulation activities for studying during an 
interview and claim that they use them far more frequently than when 
responding to a Likert item about the frequency of using "other” [than 
SR] strategies. The current results are consistent with other hndings 
that suggest self-report Likert frequency-ranking strategy statements 
may not be a valid measure of what learning activities developmental 
students use and how often they use them (Deming et ah, 1994; Ley & 
Young, 1998b; Nist et ah, 1990; Young & Ley, 1997). The results from 
this study indicate that developmental learners are more likely to rec- 
ognize that they use a strategy at least occasionally than to recall an 
activity for the strategy. Evidence suggests that the recollection maybe 
inaccurate, because developmental students’ Likert responses, unlike 
regular or higher achieving students’ strategy frequencies, have not 
been related to achievement. Developmental learners therefore may 
inflate strategy frequencies on Likert items over the frequencies they 
select when recalling what strategies he or she would use given com- 
mon instructional circumstances. In this study, developmental students 
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may have reported more accurate self-regulation information about 
self-regulation processes through an interview than on Likert items. In 
an interview, the learner may be less likely to generate a strategy that 
they did not use than to indicate using a strategy at least once that they 
can recall ever using at any time. 

The current results do not support the validity of either of the two 
self-report measures of self-regulation among developmental students 
although they do provide additional evidence that the Likert items may 
be especially problematic. Agreement between the two measures is not 
an indicator that either is valid; neither maybe. Disagreement between 
the rank order frequencies indicates the two measures are not from 
the same frequency distributions and therefore are not measuring the 
same location parameters. In this case, the interview maybe measuring 
recalled activities and frequencies but not actual activities used with 
the frequency indicated. Asking developmental students what they do 
to self-regulate their own learning processes and how frequently they 
engage in the respective activities maybe a difficult questions for poorly 
self-regulated learners to answer accurately and even more difficult 
when they are given socially acceptable options from which to choose 
a frequency such as on the Likert. 

One form of self-report may more accurately measure self-regulation 
by asking students to record what they have done during a specihc and 
relatively short instructional time, e.g., a week. This technique of log- 
ging study activities was an effective self-regulation intervention that 
increased achievement among college statistics students (Lan, 1996). 
Indeed researchers may want to investigate the usefulness of self- 
regulation interventions without attempting to measure learners past 
self-regulation activity use or predict future use given an instructional 
circumstance. 

Measuring self-regulation accurately among those who are not able or 
willing to self-regulate is fraught with problems and remains a challenge. 
Of two other possible options for measuring self-regulation activities, 
observation or self-report about self-regulation activities used during a 
very recent and relatively short instructional time period, observations, 
in this case by teachers, have also been seriously flawed (Zimmerman 
& Martinez-Pons, 1986). The remaining option, another form of self- 
report, avoids the inherent flaws of asking respondents to speculate 
about future possible activities or asking them to recall and summarize 
self-regulating activities and estimate the frequency with which they 
use the activities over a long and ill-dehned period of time. Asking the 
learners to identify what behaviors they have used during a short and 
specihed time period rather than speculating about what they might 
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use in a particular circumstance, like the interview, or what they have 
typically used without reference to a particular class, such as the Lik- 
ert items, may provide a more accurate picture of what self-regulation 
activities developmental students do or do not use and how often they 
use them. This self-regulation recording activity by regular achieving 
learners increased their achievement leading Lan (1996) to conclude 
that their self-observation led to an increase in the self-observed activity. 
Reporting self-regulation activities may increase self-regulation activity 
and achievement. 

Researchers who are investigating self-regulation with developmental 
college students should consider identifying effective self-regulation 
interventions and measuring cognitive and affective outcomes instead 
of attempting to measure interim self-regulation processes. Further 
investigations to identify a more valid and reliable self-regulation mea- 
sure for poorly self-regulating learners would seem warranted. On the 
other hand, such investigations will be challenging given developmental 
learners’ inability to reliably recall what self-regulation activities they 
do and do not do. 
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Appendix A 

The Self-Regulation Learning Seale (from Lan, 1998. Used with 
permission.) 

Below is a list of learning strategies students employ when they are 
studying. Circle the answer under each of the strategies to indicate how 
often you use the strategy when you study. An example is provided for 
each of the strategies in the parenthesis to help you understand these 
strategies. 

l.Self evaluation of the quality of work (checking one’s own 
assignments) 

always often sometimes rare never 
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2. Organizing and transforming (make an outline before writing 
a paper) 

always often sometimes rare never 

3. Goal-setting and planning (plan activities in advance) 

always often sometimes rare never 

4. Seeking information (gather information prior to beginning 
assignment) 

always often sometimes rare never 

5. Keeping records and monitoring (take notes of discussion) 

always often sometimes rare never 

6. Environmental structuring (have a specihc place to study) 

always often sometimes rare never 

7. Self consequences (arrange for rewards or punishment for 
success or failure) 

always often sometimes rare never 

8. Rehearsing and memorization (repeat material until you have 
learned it) 

always often sometimes rare never 

9. Seeking assistance from teachers/ experts (if I need help I ask 
the teacher) 

always often sometimes rare never 

10. Seeking assistance from classmates (if I need help I ask 
classmates) 

always often sometimes rare never 

11 . Seeking assistance from others (if I need help I ask oth- 
ers) 

always often sometimes rare never 

12. Reviewing tests (I review my test) 

always often sometimes rare never 

13. Reviewing notes (I review my notes) 

always often sometimes rare never 

14. Reviewing texts (I review the book) 

always often sometimes rare never 

15. Other (use other strategies not listed here) 

always often sometimes rare never 

Appendix B 

Self Regulated Learning Interview Sehedule (from Zimmerman & 
Martinez-Pons, 1986. Used with permission.) 

1 . Check the students name before he/ she enters the interview 


room. 
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2. Greet the student as follows; “Hello, I am from the graduate 

sehool at . You are [mention the 

student’s name. When he/she responds in the affirmative, 
proceed]? Thank you for agreeing to help us in this study. We 
are looking at the way students learn . " 

3. Give students the personal consent and biographical data 
forms to complete. Say, “Please read and eomplete the eonsent 
form and the biographieal data on the baek . " 

4. Continue, “I'd like to ask you some questions eoneerning the way 
you study. You will need to refer to this eard [point to the consis- 
tency scale 3x5 card]_to respond to some of the questions. Feel 
free to look at it when the time eomes. Are you ready?" 

5. For each question below: a) ask the question, b) follow up, 
and c) assign a score according to the following scheme: 

a. — Ask the question 

If the student gives a clear method, ask, “Is there anything 
else you do?" 

If the student says “YES”, but does not give an answer ask 
“What do you do?" 

If the student offers a general or ambiguous method, ask 
“Could you be more speeifie?" 

If the student says “I don’t do anything (more)” or words to 
that effect, go to b. 

b. — Then ask the follow-up question, “What if you are having 
diffieulty? Is there any partieular method you use?" 

If the student gives a clear method, ask, “Is there anything 
else you do?" 

If the student says “YES’” but does not give an answer, ask, 
“What do you do?" 

If the student offers a general or ambiguous method, ask, 
“Could you be more speeifie?" 

If the student still says "I don't do anything more" or words 
to that effect, go to c. 

For each strategy mentioned, record the question number in the 
“Item number” column on the Individual Interview Response 
Form. Record the strategy in the “Strategy” column on the 
IIRF. Use only one strategy per line. 

c. — Say, “Now, for eaeh strategy that you mentioned, I will ask you 
how often you use it. look at the eard in front of you to deeide._ 
[point to the 3x5 card]_J will read the frequeney eategories 
along with you" 

For each strategy mentioned, say, _“How often do you . . . [mention 
the strategy]?” 
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(1) Seldom (2) Occasionally (3) Frequently (4) Most of the 
time 

Record the student response in the column titled "Frequency” 
on the Individual Interview Response Form, 
d. — Then go to next question. 

Questions 

I. Assume a teacher is discussing a topic with your class, 
such as the history of the civil rights movement. He or she 
says that you will be tested on the topic. Do you have a 
particular method to help you learn and remember what 
was discussed in class? 

II. Tfeachers often assign writing a short paper outside class 
on a topic such as history. Tfeachers will often use your 
score on this paper as part of your grade. In such cases, 
do you have a particular method to help you plan and 
write your paper? 

III. Tfeachers usually emphasize that mathematics requires 
great accuracy. Furthermore, students must complete 
much math work outside class, without the help of a 
teacher. Is there any particular method you use for com- 
pleting your math assignments accurately? 

IV. Most teachers give tests at the end of the semester, and 
these tests greatly determine grades. Do you have a par- 
ticular method for preparing for this type of test in English 
or history? 

V. Many times students have problems completing home- 
work assignments because there are other, more inter- 
esting things they would like to do, such as watching 
TV, daydreaming, or talking to friends. Do you have any 
particular method for motivating yourself to complete 
your homework under these circumstances? 

VI. Outside the classroom, some students hnd it easier if they 
select or set up the place where they study. Do you have 
any particular method for selecting or setting up the place 
where you study outside class? 

VII. With homework assignments such as writing reports for 
science class or papers in English, do you have a particular 
method for checking your work after it's completed? 

VIII. When taking tests in English, science, or history, do you 
have a particular method for making sure your answers 
are correct before turning in the paper? 
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